| |
|
|
|
Robots.txt is a visual editor for Robot Exclusion Files and a log analyzer software. It allows a user to quickly and easily create the robots.txt files required to instruct search engine spiders, which parts of a Web site are not to be indexed and made searchable by the general Web public and then to identify spiders, which do not keep to those instructions.
The program provides the user a way to log onto his FTP or local network server and then select the documents and directories which are not to be made searchable.
By means of this program you will be able to visually generate industry standard robots.txt files; identify malicious and unwanted spiders and ban them from your site; direct search engine crawlers to the appropriate pages for multilingual sites; use robots.txt files for doorway page management; keep spiders out of sensitive and private areas of your Web site; upload the correctly formatted robots.txt files directly to your FTP server not switching from Robots.txt Editor; track spider visits; create spider visits reports in HTML, Microsoft Excel CSV and XML formats and more. Free program`s updates and upgrades unrestricted in time; unlimited number of Web sites to work with |
| « read more about Robots.txt 2.0 |
|
|
RECOMMENDED DOWNLOADS IN THIS CATEGORY
|
|
|
|
| » Javascript Menu Builder IRIS 1.0
|
| Javascript Menu Builder is a powerful Javascript-producing design tool for creating menu system with special effects on Web pages. Javascript can be customized highly and easily by using Javascript Configuration Program. You can define text, image, font, ... |
|
| USINGIT.COM | 580KB | Win 95/98/ME/NT/2000 | $29.95 | Downloads: 1504 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » PWFAQ 1.01h
|
| Build, manage, and maintain FAQ pages with no HTML coding required. PWFAQ even allows you to save your Metat Tags, FAQ questions, Header text, and footer text files for reuse. Use this feature to quickly add new FAQ pages or to update existing pages. Even ... |
|
| PeachWare Software | 967KB | Win 95/98/ME/2000/XP | $19.95 | Downloads: 1485 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » Html Injector 1.5
|
| Stop wasting your valuable time doing repetitive html updates! Html Injector can help you make complex changes to your site in minutes, saving you hours of valuable time. Html Injector greatly reduces the work time associated with common web site mainten ... |
|
| GX-Software | 2540KB | Win 95/98/ME/2000/XP | $14.99 | Downloads: 1512 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » Ter-Net Smart Builder 2.0
|
| Ter-Net Smart Builder is new technology used to design Web sites with a dynamic user interface, especially user interfaces for Web-applications. Smart Builder is a JavaScript library of DHTML navigational elements such as pull-down menus, expandable tree ... |
|
| Ter-Net Ltd | 175KB | Mac PPC | $29 | Downloads: 1501 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » AE Posting Acceptor Library 2.0
|
| AE Posting Acceptor Library implements server-side support for file uploading in Microsoft Internet Information Server. AE Posting Acceptor library realizes some useful features for intranet or extranet use. With AEPA you may: embed file accepting and ... |
|
| Alexander Eltsyn | 417 Kb | Windows 95/98/NT/2000 | | Downloads: 1659 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » ASPEdit 2000 2.05 Build
|
| ASPEdit 2000 is an editor for creating Active Server Pages and HTML. ASPEdit offers drop-down menus for inserting statements and functions, including ASP statements, server variables, server statements, VBScript statements, ASP remarkes, and more. It h ... |
|
| Tashcom Software | 5.69 MB | Windows 95/98/NT/2000 | | Downloads: 3730 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » ASP Beautify 1.0
|
| ASP Beautify cleans up and formats Active Server Page files. It properly indents VBScript and HTML code using either spaces or tabs. You can use it to beautify the contents of the Clipboard, or you can beautify one or more files. ... |
|
| Paessler Tools | 156 Kb | Windows 95/98/NT/2000 | | Downloads: 3803 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » Agile ColorWeb 1.01
|
| Agile ColorWeb makes picking the right colors for your Web pages quick and easy. Ensure that the colors you pick will display correctly when viewed by people using different browsers on both Windows and Mac platforms. Works with any HTML editor. Easy conv ... |
|
| Agilic Corporation | 1248Kb | Windows 95/98/NT/2000 | $10 | Downloads: 1594 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » Team Remote ASP Debugger 6.06
|
| Team Remote ASP Debugger enables your team to easily edit and debug Active Server Pages (ASP code errors) remotely and locally. JavaScript debugging and VBScript debugging is fully supported for simple and complex local, remote and team remote debugging s ... |
|
| Spline Technologies | 577KB | Win 2000/XP | $50 | Downloads: 1572 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » Art HTML Listing 3.26
|
| Art HTML Listing creates filtered, sorted, fully customizable directory listings, indexes, menus, maps and trees in HTML, DHTML, XML, text and CSV formats, as well as the list of MP3 and image files. Supports HTML titles, icons, media types, MP3 ID3 tags ... |
|
| ZZEE | 2244KB | Win 98/ME/2000/XP | $29.95 | Downloads: 1531 |
| Date added: Sep 23, 2005 |
» Details |
|
|
|
|
| » Link Checker Pro 3.3.11
|
| Link Checker Pro is the leading solution for website analysis and the detection of broken and other problem links. Link Checker Pro combines powerful features and an easy to use interface and is robust enough to deal with corporate websites containing 100 ... |
|
| Link Checker Pro | 5056KB | Win 95/98/ME/NT/2000 | $75.00 | Downloads: 1585 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » Crossword 7.50310-2110
|
| Like Scrabble? Wish you could play with family and friends who are miles away? Now it`s possible with the online Crossword script. Great, realistic, Scrabble-style action Letter bag, personal tray, password protected Automatic turn sequencing and email up ... |
|
| CrookedBush.com Inc. | 83.54KB | Linux | $24.96 | Downloads: 1470 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » MetaDataMiner Catalogue PRO 4.2.12
|
| Extract metadata from MS Office, PDF, HTML, Adobe files, IPTC info from jpeg and tif images into XML, CSV, Word files or HTML pages without programming. Set file properties via mass update function for MS Office and NTFS Windows 2000 files. You will ea ... |
|
| Soft Experience | 2995KB | Win 95/98/ME/NT/2000/XP | $175.00 | Downloads: 1392 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » AscToHTM 5.0
|
| AscToHTM is a powerful and highly configurable utility that converts text files into HTML pages. The software uses advanced text-recognition algorithms to detect the headings, bullets, indentation levels, emphasized text, tables and ASCII art in the origi ... |
|
| JafSoft Limited | 1891KB | Win 95/98/ME/NT/2000/XP | $30.00 | Downloads: 1693 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » SWC Repository 1.3
|
| SWC Repository is a tool for advanced usersof Macromedia Flash MX 2004 Professional,allow you to: -= With ease manage huge collections of SWC files(SWC Component package). -= View and edit SWC file content. -= Share your components with friends and colle ... |
|
| ShurikSoft | 1023KB | Win ME/NT/2000/XP | $20 | Downloads: 1613 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » www-Sharp 1.40
|
| www-Sharp is program for wysiwyg web-design and creating server-side and client-side HTML applications. It includes powerfull HTML editor, XML editor, tools for creating HTML help applications, .Net IDE, viewers for Windows Management Instrumentation (WMI ... |
|
| Viatcheslav V. Vassiliev | 8636KB | Win 98/ME/2000/XP | $125 | Downloads: 1492 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » SSEditor 0.9.8.402
|
| SSEditor features include: Edit any type of text file, include UNICODE text file. Support multi-file editing, edit many files at the same time in one SSEditor window. Support Drag and Drop files from the Explorer. Embedded File Explorer, you can browse a ... |
|
| Skyweb Software | 1372KB | Win 98/ME/NT/2000/XP | $29.95 | Downloads: 1463 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » Techvision Color Picker 1.2
|
| Techvision Color Picker can pick colors from anywhere on your screen with a single mouse click or a key press. After picking it shows you the color in a variety of ways, including RGB and HSL pairs, Delphi, C++ or HTML constants. With a single click you c ... |
|
| Technical Vision Software Limited | 291KB | Win 95/98/ME/NT/2000/XP | $14.95 | Downloads: 1472 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » XDenSer HTTP Server Component Pack 1.1b3
|
| With XDenSer HTTP Server Components Pack you can easily create a standalone HTTP server w/o any HTML coding. Just put TxdsHTTPServer component and TxdsFormContentProducer on your form, create 1 HTTPAction Item and your form is turned into web form ready t ... |
|
| XDenSer Components | 701KB | Win 95/98/ME/2000/XP | $34 | Downloads: 1486 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » ApoServer 1.0
|
| Platform for e-commerce and web-development, providing implementation of component oriented technologies. Using ApoServer you can develop Web-enabled application without knowing much about ApoServer utilized technologies, and provide high product quality ... |
|
| ApoSoft | 3186KB | Win NT/2000/XP | $699 | Downloads: 1579 |
| Date added: Sep 23, 2005 |
» Details |
|
|
|
|
| |
